

clear

* * * * First download the replication dataset "Manipulation_checks_replication_data.dta" from the JEPS Dataverse site

* * * Use the path to the folder in which you uploaded the data as directory

cd "/THE PATH TO THE FOLDER/"

* * * Open the data

use Manipulation_checks_replication_data




* * * * * Compute the results of the paper 

* * * * Main manuscript

* * * Proportion of subjects who failed/succeded the manipulation check

tab Experimental_treatment Response_manipulation_check, cell
* Interpretation: 27.58% + 35.13% of subjects succeeded 

* * * Figure 1: Share of subjects by experimental treatment and by response to the manipulation check question (with 95% Confidence Interval)

* * Proportion of subjects in each treatment (with 95% Confidence Interval)

proportion  Experimental_treatment

* * Proportion of subjects by response to the manipulation check (with 95% Confidence Interval)

proportion  Response_manipulation_check

* * To create the figure, copy the statistics from the previous commands, here:            (no easy to use graph command to plot proportions and CIs to my knowledge)

* For the proportion of subjects in each treatment (with 95% Confidence Interval): 

clear
input float(n Experimental_treatment prop se lower upper)
1 0 .5018992   .0079575      .4863012    .5174936
2 1  .4981008   .0079575      .4825064    .5136988
end

* Plot the statistics

twoway (bar prop Experimental_treatment) (rcap lower upper Experimental_treatment) , ysc(r(0 .5)) name(prop1, replace)

* For the proportion of subjects in each treatment (with 95% Confidence Interval): 

clear
input float(n EXP2_Eco prop se lower upper)
1 0   .2984694   .0073095      .2843391    .3129948
2 1   .2683673   .0070782      .2547191    .2824697
3 2   .4331633   .0079153      .4177152    .4487424
end

* Plot the statistics

twoway (bar prop EXP2_Eco) (rcap lower upper EXP2_Eco) , ysc(r(0 .5)) name(prop2, replace)

* Combine the two plots

graph combine prop1 prop2, ycommon 


* * * Back to the raw data
clear
use Manipulation_checks_replication_data


* * * Two-way Anova: Relationship between perception of the economy and responses to the manipulation check question

anova Perception_eco Response_manipulation_check

* * * Figure 2: Share of responses to the manipulation check question depending on the perception of the national economy prior to the experiment (with 95% Confidence Interval)

* * Modelize the probability of each response to the manipulation check question based on a multinomial logistic regression

mlogit Response_manipulation_check i.Perception_eco

* * Compute the predictions and confidence intervals from each response to the manipulation check question

margins Perception_eco, predict(outcome(0)) saving(margins1, replace)
margins Perception_eco, predict(outcome(1)) saving(margins2, replace)
margins Perception_eco, predict(outcome(2)) saving(margins3, replace)

* * Plot the predictions

combomarginsplot margins1 margins2 margins3


* * * T-test: Relationship between the experimental treatment and subjects' perception of the national economy prior to the survey experiment

* * With all subjects

ttest Perception_eco, by(Experimental_treatment)

* * Excluding subjects who failed the manipulation check

ttest Perception_eco if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental_treatment == 1 & Response_manipulation_check == 2), by(Experimental_treatment)

* * * Figure 3: Average perception of the national economy prior to the experiment depending on the experimental treatment (with 95% Confidence Interval)

* * Modelize the perception of the national economy depending on the experimental treatment for all subjects based on a linear regression

reg Perception_eco i.Experimental_treatment 
margins Experimental_treatment
marginsplot, ysc(r(4.5 5.5)) name(marginsa, replace) ylabel(4.5(.2)5.5) xsc(r(-.5 1.5)) recast(bar) title("Including subjects who failed "" the manipulation check") ytitle("Perception of the national economy" "prior to the treatment")

* * Modelize the perception of the national economy depending on the experimental treatment excluding subjects who failed the manipulation check based on a linear regression

reg Perception_eco i.Experimental_treatment if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental_treatment == 1 & Response_manipulation_check == 2)
margins Experimental_treatment
marginsplot, ysc(r(4.5 5.5)) name(marginsb, replace) ylabel(4.5(.2)5.5) xsc(r(-.5 1.5)) recast(bar) title("Dropping subjects who failed "" the manipulation check") ytitle(" ")

* * Combine the two plots

graph combine marginsa marginsb, nocopies ycommon


* * * Table 1: Results from linear regression models of the level of nostalgia

* * Model 1

reg Nostalgia i.Experimental_treatment if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental_treatment == 1 & Response_manipulation_check == 2)

* * Model 2

reg Nostalgia i.Experimental_treatment Perception_eco if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental_treatment == 1 & Response_manipulation_check == 2)

* * Model 3

reg Nostalgia i.Experimental_treatment 




* * * * Online appendix

* * * Table 1: Results from linear regression models of the level of nostalgia

* * Model 1

reg Nostalgia i.Experimental_treatment i.Country_num if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental_treatment == 1 & Response_manipulation_check == 2)

* * Model 2

reg Nostalgia i.Experimental_treatment Perception_eco i.Country_num if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental_treatment == 1 & Response_manipulation_check == 2)

* * Model 3

reg Nostalgia i.Experimental_treatment i.Country_num 


* * * Figure 1: Share of subjects who failed the manipulation check depending on the time they spent on the survey (with 95% Confidence Interval)

* * Extract subjects' decile of survey duration based on the total survey duraction in seconds

egen Deciles_survey_duraction =xtile( Survey_duration ), nq(10)

* * Generate a variable capturing Failure to the manipulation check

gen Failure_manipulation_check = 1
replace Failure_manipulation_check = 0 if (Experimental_treatment == 0 & Response_manipulation_check == 0) | (Experimental_treatment == 1 & Response_manipulation_check == 2)

* * Modelize the proportion of subjects who failed the manipulation check depending on the time they spent on the survey based on a logistic regression

logit Failure_manipulation_check i.Deciles_survey_duraction
margins Deciles_survey_duraction

* * Plot the predictions

marginsplot






